Vanishing point detection with convolutional neural networks

In a graphical perspective, a vanishing point (VP) is a 2D point (in the image plane) which is the intersection of parallel lines in the 3D world (but not parallel to the image plane). In other words, the vanishing point is the spot to which the receding parallel lines diminish. In principle, there can be more than one vanishing point in the image. VP can commonly be seen in fields, railroads, streets, tunnels, forest, buildings, objects such as ladder (from looking bottom-up), etc. It is an important visual cue useful in several applications (e.g., camera calibration, 3D reconstruction, autonomous driving). Inspired by the finding that vanishing point (road tangent) guides driver’s gaze [1, 2], in our previous work we showed that vanishing point attracts gaze during free viewing of natural scenes as well as in visual search [3]. We have also introduced improved saliency models using vanishing point detectors [4]. Here, we aim to predict vanishing points in naturalistic environments by training convolutional neural networks in an end-to-end manner. Traditionally, geometrical and structural features such as lines and corners (e.g., using Hough transform [5]) have been applied for detecting vanishing points in images. Here, we follow a data-driven learning approach by training two popular convolutional neural networks, Alexnet and VGG, for: 1) predicting whether a vanishing point exists in a scene (on a n × n grid map), and 2) If so, we then attempt to localize its exact location.